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1. Introduction 


We introduce a novel representation of two-dimensional shape that we call smoothed 
local symmetries (SLS). Smoothed local symmetries represent both the bounding 
contour of a shape fragment and the region that it subtends or encloses. In this 
paper we develop the main features of the SLS representation &nd describe an 
implemented algorithm that computes it. The performance of the algorithm is 
illustrated for a set of tools. One step in the algorithm is the construction of a 
representation of the significant changes in curvature along the contour of the 
shape. We call that representation the curvature primal sketch [Asada and Brady 
1984], since it is analogous to the primal sketch representation of intensity changes 
advocated by Marr [1976]. We conclude by sketching a method for determining the 
articulation of a shape into subshapes. A preliminary version of this paper appeared 
as [Brady and Asada, 1984]. 

The representation of shape is a fundamental problem in computer vision that 
has been surprisingly neglected. It is becoming increasingly important in Robot 
vision as the inadequacies of currently available systems manifest themselves. So 
long as the position and orientation of objects is tightly constrained, so long as 
contrast is sufficient to allow trivial segmentation into figure and ground, and so 
long as objects do not overlap, simple vision systems suffice. Increasing uncertainty • 
in any of these variables requires a substantial increase in system sophistication. 

Fortunately, there has been considerable advance in our understanding of 
computer vision over the past decade (see [Ballard and Brown 1982, Brady 1982a, 
Marr 1982] for reviews). For example, edge finding, though far from being a solved 
problem, is increasingly reliable and able to support the accurate computation of 
curvature (see below). Better edge finders are a prerequisite to improved shape 
representations. The additional information that they provide supports jicher 
representations of shape, such as that discussed here. Much attention has focussed 
on three-dimensional vision. There has been significant progress in computing 
depth (or local surface orientation) using stereo, photometric stereo, structured 
light, and bounding contour. In other work, we build on these additional capabilities 
of computer vision to develop better representations of three dimensional shape 
[Brady and Yuille 1984]. 

A representation of shape can be judged according to its suitability for many 
uses in Robotics: 

• Recognition. Given a database of models, recognition amounts to associating 
an object with its model. There are two principal difficulties. First, objects may 
overlap. An object has to be recognised from its non-overlapped portions and 
these portions cannot be predicted in advance. The shape of each portion needs 
to be represented, and provides constraint on model identification. The richer the 
representation of a visible fragment, the tighter the constraint. Recognition involves 
the summation pf the constraints* from the separate visible portions. It is best 
suited to a highly parallel computer such as the connection machine [Hillis 1981]. 
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Representing a shape by a set of global features such as its moments cannot deal 
with overlap. 

Second, in general, the database of models must allow for variations in shape. 
For example, not all tack hammers are exactly alike. Also, tack hammers are 
clearly distinguishable from Warrington hammers and ball peen hammers, but they 
are equally all clearly identifiable as hammers. Artificial Intelligence has developed 
semantic networks to represent classes and subclasses of objects and their subparts. 
In particular, Brooks and Binford [1980] -developed ACRONYM to address such 
problems. This issue will be taken up again in Section 6. 

• . Inspection. Since a part to be inspected may have defects, it is unreasonable to 
expect a perfect match between a shape description and its model. We need to be 
able to recognize the model corresponding to a flawed object and to determine the 
nature of the flaw. The richer the representation of shape that can be computed for 
an object, the greater the possibility of automatically generating an explanation of 
the flaw from the mismatch between the object description and its model. This has 
been a recurring theme in Artificial Intelligence [Winston 1984]. Flaws are typically 
restricted to a local portion of a shape. The shape representation needs to be stable 
with respect to local changes, yet sensitive enough to represent them [Marr and 
Nishihara 1978]. We say that the shape representation must have local support. 

• Grasping. Part of our understanding of a tool such as a screwdriver is how 
to pick it up, and how to grasp it when we use it. For a given end-effector, one 
can imagine motor program links in the model of an object that contain such 
information. However, faced with an object that we have never seen before, we still 
grasp it sensibly. In order to do this, we must determine a grasp position from a 
representation of the shape of the object to be grasped. Boissonat [1982] and Brady 
[1982b] have suggested ways in which this might be done. 

• Reasoning. Robotics is the intelligent connection of perception to action. 
Intelligent robots need to be able to reason about the tasks that they are to 
perform, for example to plan processing and assembly sequences. Currently popular 
models of reasoning, such as expert systems, are typically supplied with pre-digested 
sensory information. Shape representations should support enhanced reasoning (see 
for example [Brady 1984b]). 

In previous papers [Brady 1982b, 1982c, 1983] we have introduced a 
representation of two-dimensional shape that we call smoothed local symmetries 
(SLS). Section 2 summarizes the main ideas, and then develops a parametric analysis 
of the SLS of a shape. The parametric description is both region and contour based. 
In Section 3, we describe an implementation of the representation, illustrating the 
performance of our programs in Section 5 using a set of hand tools. We discuss in 
Section 4 the reasons for computing a representation, called the curvature primal 
sketch , of the significant curvature changes along a contour. Asada and Brady 
[1984] discuss the curvature primal sketch in more detail. Finally, in Section 6 we 
sketch a method, currently being implemented, for determining the articulation of 
a shape into subshapes. The emphasis of this paper is on competence: we develop 
an implemented representation that can support a variety of applications without 
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excessive concern for ” real time” performance. Hardware implementation of the 
ideas developed here will be discussed elsewhere. 


2. Smoothed local symmetries 


2.1. Background on shape description 

Shape is represented in most current robotics or commercially available vision 
systems [Agin 1980, Holland, Rossol, and Ward 1979, Perkins 1978, Rosenfeld 1982] 
in terms of global features such as*, the center of area, number of holes, the aspect 
ratio of the principal axes, and the ratio of the perimeter squared to the area. 
Such features can be computed efficiently and they are reasonably insensitive to 
noise and quantization, even for low resolution binary images. Unfortunately, the 
computed value of a global feature for the visible portion of an occluded object 
bears an arbitrary relationship to the value that would be computed for the whole 
object. It follows that it is at best difficult, and usually impossible, to recognize 
occluded parts using global features. 

Bolles and Cain [1982] uses “focus features” to handle limited cases of occlusion. 
An object model is a graph whose nodes are features such as corners and holes and 
whose edges are the exact distance and relative orientation of the two nodes related 
by the edge. Recognition is implemented by finding maximal cliques in model 
graphs. In essence, Bolles’ representation is a compromise between global features 
and representations that have local support, such as smoothed local symmetries. 
The representation is, however, metrically exact and uses only simple features such 
as rectangular corners, holes, and their relative position. It is restricted to straight 
line contours. 

Nearly every representation of shape can be classified as being either contour 
based or region based. Contour and region based representations of shape make 
different aspects of shape explicit. Contour based representations encode the 
bounding contour of a shape. Examples include Fourier series expansions of the 
contour, chain encoding, and spline approximations (see Pavlidis [1978]). Asada and 
Brady [1984] have developed a representation that makes explicit the significant 
curvature changes along a contour. Attneave [1954] showed that such changes, 
essentially the curvature analogue of the primal sketch (see [Asada and Brady 
1984]), carry a great deal of information for the human visual system. Certainly, 
the human visual system is remarkably sensitive to curvature changes [Watt and 
Andrews 1982]. It is difficult, however, to compute or determine important regional 
properties, such as symmetry, or descriptions such as “elongated and curved”, in 
a purely contour based representation of shape. 

Region based approaches represent a shape by encoding the two-dimensional 
space occupied by the shape. Examples include quadtrees [Rosenfeld and Kak 1982], 
the symmetric axis transform (SAT) [Blum and Nagel 1978], and (two-dimensional) 
generalized cones [Nevatiaand Binford 1977, Brooks 1981]. The SAT and generalized 
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Figure 1. a. A hammer claw. b. Its geometric representation. The solid lines amount to a contour 
based representation, while the dotted lines encode the region occupied by the claw. _ 

cylinders emphasize elongation, describing a shape by a one-dimensional axis and 
the disposition of the shape about that axis. Occlusion poses a difficult problem 
for purely region based representations, since recognizing partly occluded shapes 
requires a representation that has local support and a parametric description that 
is in part contour based. The smoothed local symmetries representation, described 
in the next section, is both contour based and region based. 

2.2. Definition of smoothed local symmetries 

Figure 1 shows a hammer claw and its geometric representation. The arc AB is 
described as a convex arc whose curvature is constant. In our implementations, arcs 
are described either by circles or by Cornu spirals, for which curvature varies linearly 
with arc length. The total angular change along AB is recorded. The discontinuity 
in curvature at B is made explicit. There are various kinds of primitive curvature 
discontinuities: in this case, B is an acute angled corner. Other primitive curvature 
discontinuities are defined below. They include ends, cranks, smooth joins, bumps , 
and dents. The arc BC is concave, and is also described by the best fitting 
circle. These descriptors form a contour based representation. The region based 
description is shown between the representations of AB and BC. It is described as 
a smoothed local symmetry of primitive type beak. Other primitive region types 
are defined below. They include cup, sector, wedge, plinth, and flare. The region 
description has several parameters associated with it, including a measure of the 
width of the region, and the curvature of an associated axis. In this case, we note 
that the width decreases monotonically in the direction approaching B, and has an 
axis that “slowly” curves to the right. Curvature is quantized to 5 levels, following 
Hollerbach [1975]. For a fuller account of parametric descriptions, see Heide [1984]. 

The definition of smoothed local symmetries is in three parts. First, Figure 
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Figure 2. a. The geometry of a local symmetry, b. The point A has local symmetries with 
points Bi and B^. 

2a shows the geometry of a local symmetry. A and B are points on the bounding 
contour of a shape. The angle a between the line BA and the outward normal n* 
at A is the same as the angle between BA and the inward normal njg at B. (A 
precise definition is given in the next section.) Figure 2b illustrates that in general, 
there may be several points B{ forming local symmetries with a given point A. 

Implicit in the definition of local symmetry is the requirement that the bounding 
contour of the shape be extracted and the tangent angle computed sufficiently 
accurately. We use the edge finder developed recently by Canny [1983]. This 
extremizes the product of the signal-to-noise response at a step edge and a measure 
of the localization of the edge, while producing a single response to a single Step. 
Earlier work used a non-directional operator based on a difference of Gaussians 
filter, proposed by Marr and Hildreth [1980]. The Canny operator is preferred 
because it gives more accurate tangent directions and is less susceptible to noise. 
For more details, see [Canny 1983]. 

Second, we consider the loci of local symmetries that are maximal with respect 
to forming a smooth curve. We call such loci axes or spines. Figure 3a shows the 
axis for a portion of a contour that contains a corner. Each axis is an alternative, 
locally plausible way to describe some piece of the contour and the region subtended 
by the axis. We call this portion of the shape, consisting of a part of the contour 
and a subregion of the shape, the cover of the axis. Notice that the covers of certain 
axes are wholly, that is, properly, contained in the covers of others. In such cases, 
we say that the axis with the contained cover is subsumed by the axis with the 
containing cover. Figure 3b shows the axes for a rectangle. The short, diagonal axes 
are subsumed by the horizontal and vertical axes, that are the global symmetries. 
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Figure 3. a. The smooth locus of local symmetries for a portion of contour containing a 
corner, b. The candidate axes for a rectangle. The small diagonal candidate axes are subsumed, 
leaving the (global symmetry) axes as smoothed local symmetries, c. The smoothed local symmetries 

of a square. ____ 

Notice that in case the rectangle were a square, the diagonal axes would not be 
subsumed (Figure 3c). 

Figure 4 shows some examples of smoothed local symmetries. The smoothed 
local symmetries are indicated by their axes. It should be remembered that such 
axes are merely graphic illustrations , since they do not show the set of parameters 
and symbolic description that are associated with the smoothed local symmetry 
(see Figure 1 and Section 2.3). 

Originally, Brady [1983, page 56] suggested that subsumed smoothed local 
symmetries should be deleted. Instead, we now accord them lesser importance in 
a hierarchical representation of the shape. The diagonal axes in Figure 3b, for 
example, provide information about the corners of the rectangle, which, though less 
important than the global symmetries of the shape, still convey useful information. 
For example, Figure 5a shows an acute angled corner discontinuity on a variety 
of contour fragments. The differences between the contour fragments is explicitly 
represented by region descriptors such as beak (see below). Figure 5b shows an 
occluded instance of a corner. The actual curvature discontinuity is not present, 
but it is implied by the monotonically decreasing width of the region and its small 

absolute value. 

Smoothed local symmetries were developed from the ideas underlying the SAT 
and generalized cylinders. Brady [1983, pages 53 - 56] discusses the need to make 
axis smoothness explicit and to refine the definition of local symmetry implicit in 
the SAT. Most importantly, all of the local symmetry axes are made explicit for 
smoothed local symmetries. 

The SAT is defined semi-constructively and topologically as the union of the 
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Figure 4. Examples of smoothed local symmetries. _____ 

centers of maximal disks that touch at least two points on the bounding contour 
of a shape. Figure 6 defines normal, branch, and end points of the SAT. The SAT 
is a piecewise smooth, single, forked spine. It is not obvious a priori that the SAT 
is piecewise smooth; but that has been shown by Bookstein [1979]. Branch points, 
where the maximal circle touches the contour in more than two points, are a major 
cause of the problems of the SAT, giving particularly poor responses when' there 
is a discontinuity on the bounding contour of a shape. Figure 7 shows the SAT of 
a rectangle. Note that, unlike the smoothed local symmetry representation (Figure 
3b), neither of the global symmetry axes are found. Blum and Nagel [1978, page 169] 
note that “the SAT is not the simplest description for rectilinear figures”. Branch 
points were originally introduced into the definition of the SAT as a discriminant 
for isolating sub-shapes (the name comes from finding branches off the trunk of a 
tree). They do not perform that task reliably either. Some other differences between 
the SAT and smoothed local symmetries are presented in [Brady 1983]. 

Figure 8 compares smoothed local symmetries with the SAT. Figure 8a shows 
a triangle. The SAT makes explicit the center of the inscribed circle. The smoothed 
local symmetries are the median lines. If the triangle is isosceles, a median line is 
a global symmetry. The SAT of an' ellipse is the major axis (Figure 8b), while the 
smoothed local symmetry makes both axes explicit. Figures 8c and 8d show how 
curvature discontinuities on the contour of a shape cause unintuitive branch points. 
The dotted lines are exterior portions of the SAT or smoothed local symmetries. 
As we shall see in Section 6, they* are useful for determining subshape joins. 
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Figure 5. Several curvature discontinuities that subtend the same acute angle. Differences 
between the instances is found in the region descriptors, b. An occluded corner. 



Figure 6. The symmetric axis transform. The centers of maximal circles that touch the 
contour at one, two, and more than two points are called end, normal, and branch points, 
respectively. - ___— 

2.3. Analysis of smoothed local symmetries 

Suppose that the point A in Figure 2a is ta in some vector frame. Let the 
unit vector in the direction BA be u(s) and let the length of AB be h[s), where s 
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Figure 7. The SAT of a rectangle. Note that, unlike the smoothed local symmetry 
representation shown in Figure 3b, the SAT fails to make explicit either of the global symmetries 
of the rectangle. The SAT forks at branch points that signal the corners of the rectangle. 

indicates distance along the smoothed local symmetry. The point P at which the 
local symmetry is recorded is r p = |( r A + r#). By definition of the local symmetry, 
the angle a between u and the outward normal at A is equal to that between 11 
and the inward normal at B. Aside from the global coordinate frame in which 
and r B are expressed, we introduce a moving frame with origin P, collinear with 
u, and measure angles counter-clockwise from 11. 

The tangent t A at A is given by 



dr A 

ds A ' 


Since ds A is a unit speed parameter for the contour, t A is a unit vector. 
Similarly t B is a unit vector. Suppose that a step ds along the local symmetry axis 
at P corresponds to a step ds A at A and a step —ds B at B. We use a minus sign 
to indicate a global direction of traversal of the curve. Bookstein [1979] shows that 
the steps at A and B are in opposite directions. We denote by s' A) and ^ by 
s' B . Note that s\ • s* B is negative. 

With respect to the moving coordinate frame centered on P, the tangent t B 
at B is at angle 7 r /2 - a and t A is at a - 7r/2. It follows that 


u -.(tyi - t B ) = O. 


Hence u is perpendicular to the unit vector 
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Figure 8. Comparison of smoothed local symmetries (left) and the SAT (right), a. A triangle, 
b. An ellipse, c. and d. Illustrating the deterioration of the SAT when curvature discontinuities 
cause spurious branch points. _ _ •_ 

We assume that |a| < tt/ 2 for a local symmetry. In general, t is not the tangent 
t p to the smoothed local symmetry at P since 


t p = 


Ts rp 


- t bs'b) 


( 2 . 1 ) 


Suppose that t p is at angle <f>[s} counter-clockwise from t, that is, at angle <f> j 
from u. Then, forming the dot product of Equation (2.1) with t gives 


IMIcos^ = (s^ + «b) 


cos a 


By forming the cross product of Eq. (2.1) with and tp, we find 
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^ m J* 

ds B r 


Figure 9. The smoothed local symmetry of two concentric circles of radii R and r. The ratio 
daji/daB is equal to R/r, and can be arbitrarily large. 


j Jtj»Xt B || 
x HtxXtall 


= 2IIMI 


sin a + ^ 
sin 2 a 


(2.2a) 


and 


J . HtfXt^ll 

B ||t B Xtx|| 


= 2IIMI 


sin a — ^ 
sin 2 a 


(2.26) 


The ratio s\ls' B — ds^/ds# can be arbitrarily large, as can be seen in Figure 9, 
which shows the smoothed local symmetry of two concentric circles, the ratio of 
whose radii is large. 

It follows from Eqs. (2.2) that 


s [4 sin a + 4> 

8 b sin a — ^ ’ 

apart from the special case (like concentric circles) when a = <f>. It follows that 

j _ j 

tanj^ = tan - j-. (2.3) 

8 A + 8 B 
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From r A -r B — h(s)u, we deduce 


h'(s) = (t^ • u)s^ + [t B • u)s'p 

= sin a(s^ + s f B ) . (2*4) 

= 2||tj>|| tan a cos <f> 

For example, if <j> — 0, as it is for a global or flexed symmetry, then h,' = 2 tan a, 
as expected. Suppose that t A is in direction <j> A with respect to some base direction, 
and t B is in direction <f > B • Since <t> A — <f>B = 2a — 7r, we find 

2a' = <t>' A — 4>'b ‘ (2.5) 

= K A s\ — *B s 'b> 

where, k a , respectively k B) is the curvature of the contour at A, respectively B. 
Similarly, the curvature of the smoothed local symmetry axis, /c p is given by 

Kp = -f -(« a s a + *b s b) (2*®) 

Of the parameters introduced so far, a, a', h, h!, k A) and k b are directly 
available. In general, ds A and ds B are small, so their computation is poorly 
conditioned because of quantization. It seems that <f> is inevitably computed 
indirectly from points P and the tangents at local symmetries. Finally, and <t>' 
involve second differential quantities. We solve the linear equations (2.4) and (2.5) 

to find 


8 A _ + 2a r sin a 

s' B ~~ h!K A — 2a' sin a * 


so by Eq. (2.3) 


tan <f> — tan a 


h'{K B - k A ) + id sin a 

h'{* A + k b ) 




Consider, as a special case, a (non-flexed) global symmetry. The axis curvature 
Kp is everywhere zero, as are <f> and <f> ( . Also, —k b = k a = say. From Equations 
(2.5) and (2.6) 


cl — k a s a = -k B 8 B . 

We find that s' A = 1/ cos a, so thatj|t/>|| = 1, hence h' = 2 tan a, and cl = */cos a. 
It follows that extrema of h occur when a is zero and they provide natural places 
to segment a symmetric object into its components. The points where a 0 do 
not coincide with extrema of k a in general. Figure 10 illustrates this. It shows a 
fragment of a symmetric shape whose curvature consists of a Cornu spiral smoothly 
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Figure 10. Segmenting a symmetric shape. The contour is the smooth join of a Cornu spiral 
and a circle. The inflexion point coincides with the curvature maximum of the upper part of the 
contour. The minim um width occurs where a is tero. ___— 

joined at an inflexion to a circle. The inflexion points are marked. The curvature of 
the upper part of the contour attains a maximum at the inflexion point. However, - 
the shape fragment is more naturally segmented where h is smallest (compare 

[Hollerbach 1975]). 

As a second special case, consider a worm [Blum and Nagel, 1978], for which 
h is constant (Figure 11). It follows that ti = h" = 0. and that a = <f> = </>' = 0. 
We find that 


K P 

PA + PB 

2 

PB + h 

where p = - denotes radius of curvature. 

■1C 

Not surprisingly, the description of a worm amounts to the description of a plane 
curve that is the axis, and a statement that the cross-section function is constant. 
More generally, consider a flexed symmetry [Blum and Nagel, 1978]. It is easy to 
show that a shape is a flexed symmetry if and only if <t> = 0. The shape description 
is neatly factored into (a) a description of the shape of the flexed symmetry 
axis, (b) a description of the shape orthogonal to the flexed symmetry. That is, 
in the special case of a flexed symmetry, smoothed local symmetries coincide 
with (two-dimensional) generalized cones. In general, however, the smoothed local 
symmetry representation accords more closely with human perception than do 
generalized cones (see for example' [Brady 1983, Figure 10]). 



PP — 
PA = 
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Figure 11. A flexed symmetry. _________ 

Indeed, it is clear that ^ is a local measure of the lack of symmetry of a shape. 
This is because the angle between and tp is a — <j> t while the angle between 
t B and tp is a + </>. The difference between these two angles is 2<f>, and is zero 
for a flexed symmetry. Generalized cone parameterizations are not well suited to 

describing asymmetric shapes. 

Interestingly, just the signs of ti, k a , k b , and a' allow us to .define more than 
the basic shapes proposed by Blum and Nagel [1978]. Figure 12 assigns symbolic 
descriptors to primitive shapes, defined by the signs of the flanking curvatures. 
The beak primitive occured in Figure 1. Brady [1983, page 62] compares the 
descriptors with those proposed by Blum and Nagel [1978). The primitive shapes 
have associated descriptors, such as a' and h', which define, for example, the rate 
of flaring of a flare shape. Heide [1984] has investigated the computation of these 

symbolic descriptors. 

2.4. An 0(n 2 ) algorithm for computing smoothed local symmetries. 

We have constructed one algorithm to investigate the competence of the 
smoothed local symmetry representation without worrying unduly about perfor¬ 
mance. The next section describes a more practical algorithm we are developing in 
a parallel effort. The time complexity of the algorithm described in the remainder 
of this section is 0(n 2 ), and consists simply of testing every contour point against 
every other to find local symmetries. 

More precisely, consider fixing contour point A in Figure 13, and testing 
whether the point B forms a local symmetry with A. As in the previous section, we 
set up a local coordinate frame BA and measure angles counter clockwise. Denote 
the unit vector in the direction BA by u(t), where t denotes arclength measured at 
B, and let the angle between u alid the outward normal n B be p. B forms a local 
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Figure 12. Symbolic descriptors for primitive shapes defined by the signs of the curvature 
at the points A and B defining a local symmetry. The sign of h! defines the direction of the shape 
primitive. The values of a ' and b! estimate the rate of increase of the cross section of the shape. 

symmetry with A if and only if tt — 0 *= a, which is if and only a + 0 *r = 0 . 
Define the function f(t) = a(t) + 0{t) - ?r. More properly, / should be denoted f A 
to show that A is fixed. Finding local symmetries of A corresponds to finding zero 

crossings of f(t). 

The simplest version of the algorithm considers every point A , and tests every 
point B as a local symmetry with A. Figure 14 illustrates two problems that cause 
us to include in the implementation of local symmetry two cut-off tests at A and 
B. The low curvature of the contour between A and B in Figure 14a gives rise to 
the tiny candidate axis shown. In practice, there are many such tiny axes along 
the contour,' and so we suppress them by requiring the angle between the normals 
at A and B to be above a threshold. In Figure 14b the curvature at point B is 
large, and so a small portion of the contour at B gives rise to local symmetries 
along a relatively long portion of the contour at A. We currently solve this problem 
by requiring the length |Z?A| to be no greater than the radius of curvature at 
points A and B. Figure 15 shows some examples of candidate axes produced by the 
algorithm. 
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Figure 13. Geometry for determining a local symmetry in the 0(n 2 ) algorithm. 

A 


(0) (b) 

Figure 14. a. Local symmetries in low curvature portions of the contour, b. Local symmetries 
from a high curvature portion of the contour. _____ 

One way to make the 0(n 2 ) algorithm more useful in practice is to sample 
the contour (Figure 16). This is particularly useful for long portions of the contour 
where the curvature is small. It follows from the analysis in the previous section that 
the curvature of the axis is also small. By the mean value theorem, if /(a) • /(6) < 0 
there is a zero-crossing between t = a and t = b, which can be found by binary 
subdivision. If /(c) • f(d) > 0, we cannot conclude that there is no zero-crossing 
between c and d. We can approximate / between c and d, say by a function 
Japprox) and sample more closely to find a zero crossing of / only if there is a 
zero crossing of fapprox- We can use a cubic approximation to / if we can estimate 
/'(c) and f'(d). Now /'(t) = a' + /?'. It can be shown that o' - P = -*B- Also > 
f) 1 = + (h'/h) cot/8, from which we find 





kb + 2 7 - cot 0. 
n 
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Figure 15. Examples of candidate axes found by the algorithm. 




Figure 16. Sampling the contour to increase the speed of the 0(n*) algorithm. The dotted 
line indicates an approximation to the axis computed from the sampled points. _ 

In the next section we discuss a more ambitious algorithm that uses a curvilinear 
approximation to the contour. 

An axis describes a portion of the region subtended by the shape, and does so 
by interpreting the relationship between two portions of the contour of the shape. 
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Figure 17. Example of the smoothed local symmetries found by Ilcide’s program, a. The 
axes found for the carving knife, b. The cover of the blade axis. c. The cover of the handle axis. 

Let A be a candidate axis, and denote by A & and the left and right portions of 
contour that give rise to the contour. Just as an individual point can have several 
local symmetries, so a portion of the contour of a shape can be the left or right 
portion of several candidate axes. Let Z/( be the region subtended by A. Now 
suppose that B is another candidate axis, and suppose that Ag overlaps Aj. We 
say that A subsumes B if Ag is contained in, but not equal to, A/ and if Zb is 
contained in Za- The mapping from axis to contour is in fact a diffeomorphism 
Bookstein 1979], and so subsumption can be determined from the contour. Heide 
1984, forthcoming] has implemented the subsumption operation, and demonstrated 
its use on a variety of shapes. Figure 17 illustrates the performance of Heide’s 
algorithm on an image of a carving knife. Figure 17a shows the axes of the three 
smoothed local symmetries found by the program; Figure 17b shows the cover of 
the blade, and Figure 17c the cover of the handle. 

3 . Computingsmoothed local symmetries by approximating the 
contour 

We are developing a program that computes the SLS of a shape by approximating 
the contour. One reason for doing this is that the computation of smoothed 
local symmetries can be noisy; computing smoothed local symmetries from an 
approximation to the contour can significantly reduce the noise. The basic approach 
is easily stated: choose a set of points on the contour, and then use them to 
construct a piecewise smooth approximation to the contour using the chosen points 
as “knots”. We then compute the smoothed local symmetries of the approximation 
to the contour. We currently approximate the contour using straight lines and 
circles. Figure 18 shows some example smoothed local symmetries found using the 
algorithm. 
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Figure 18. Candidate smoothed local symmetries found using the approximation algorithm, 
a. hammer, b. a pair of pliers, c. a carving knife. 

The knot points should not be chosen completely at random. Consider for 
example a corner (Figure 19). The knot should be placed at the corner rather than 
being placed on one side of it, in which case a smooth approximating curve has 
to include the discontinuity at the corner. More generally, the set of knots should 
mark significant changes in the curvature along the contour. Finding significant 
curvature changes is a difficult problem exactly analogous to computing a rich 
representation of intensity changes in one direction across an image (“the primal 
sketch” of [Marr 1976]). In the next section we sketch a method for computing such 
a representation of curvature changes. Asada and Brady [1984] give more details. 

Once we have a set of knot points, we can compute a piecewise smooth 
approximation to the contour. Asada and Brady [1984] observe that, at least 
for the class of tool shapes that we have considered so far, locating the knot 
points accurately is more important than the choice of smoothing functions to 
approximate the contour between the knots. To date, we have found that fitting 
circles between the knots is efficient and gives good results (see [Asada and Brady 
1984] for examples). 

We try to fit circles to the points {(x,-,y,)|l < i < n} between adjacent knot 
points. Suppose the circle is (x — xo)^(y — yo)^ = 0. We form the square 
error term 


E{x o, y 0 ,r) = £ {(*< - *o ) 2 + (y, - y 0 ) 2 - r 2 ) , 
and minimize E with respect to its parameters. We have 

' 2 = i{B*.-*o ) 2 + £(v.-vo) 2 }, 
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Figure 19. Knot points should mark significant changes of curvature along the contour, 
a. A corner fragment on a contour, b. Approximation when knot points (marked by crosses) are 
placed to either side of the perceived corner, c. Approximation when the corner is made explicit. 

and, after algebraic manipulation, find that two linear equations must be solved 
for zo and Vo : 


10 £ ( x i - x f + !«> £( x * - *)(v» - v) = ^ £ (*< + Vi)( x i - *) 

10 £ (x { - S)(y. - p) + yo £ (y.- - y) 2 = ^ £ (* 2 + v 2 )(y< - V) 

where x is the average of the X{ } and y the average y,\ The circular approximation 
can become poorly conditioned when the radius of the circle is large (equivalently, 
the curvature is small), in which case -the contour is better approximated by a 
straight line. Denote the matrix of coefficients of xo>VO by A: 


* _ f £(*.--1) 2 E (*.• - 2)(y< - 57) 
l£(*i-®Xw-r) E (y,- - V? J 

The condition number of the matrix A = (a tJ -) is defined (see for example 
[Strang 1980, p. 282]) to be the ratio X ma x/X m i„ of its eigenvalues. Since the 
eigenvalues of A are the roots of a quadratic in the coefficients, it is more convenient 
to work with the ratio: 

, IX1-X2I 

Xi -f X 2 _ 

\/(aii — 022 ) 2 + 4af 2 

= T JV(Aj 


20 



Bmdy&Asada 


OitlOUI/Mtu M. 


in » 



Figure 20. Geometry for local symmetry of a point P and a circular arc. The point (x e ,y c ) 
is the center of the circular arc, and R is the local symmetry of the point P and the circular arc. 

where the X t - are the eigenvalues of A. If L is less than a threshold (currently 
0.96), corresponding to the condition number of A being less than 5, we accept the 
circular approximation, else we fit a straight line to the points 

In the previous section we described a algorithm for computing the 

smoothed local symmetries of a shape. For each point along the contour we compute 
the set of local symmetries. We can use the contour approximation to speed the 
computation by a large factor. Let A be an arbitrary point on the contour. Instead of 
comparing A against all other points along the contour, we compare A against each 
of the circular arcs that approximate the contour. Figure 20 shows the geometry 
for determining the local symmetry of a point and a circular arc. The Figure treats 
the case of positive curvature k of the circle. We assume that the point A is at the 
origin and that the x-axis is aligned with the tangent at A. Let the center of the 
circular arc be (x 0 ,yo), and let the radius be r. Let the local symmetry between A 
and the point B on the circular arc occur at angle 6 measured counter clockwise 
from the radius that is perpendicular to the y axis. It can be shown that 


e 

\AP\ 


„, _i io + r + i/o 
= 2 tan - 7 —;-; 

nrn — Ir 4- */n 1 


In these expressions it is assumed that 0 < |AP| < r, which is the curvature 
cut-off condition referred to above. Similar analyses are possible in the case k < 0 
and for a straight line segment. Figure 21 shows the result of this on the image 
of a pair of pliers. The global line of symmetry is quite visible, but is noisy. The 
problem is that a single point can form local symmetries with several other points 
because of discretization and round-off. 
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We can further reduce the noise in the computation of the smoothed local 
symmetries, as well as speeding their computation, by directly comparing two 
circular arcs and computing an analytic expression for the smoothed local symmetry. 
This allows successive axes to be chained together. Figure 22a shows the geometry 
for computing the analytic form of the smoothed local symmetry of two circular 
arcs. There are various cases to consider, depending on the signs of the curvature 
of the circular arcs. In region I (see Figure 22a) we can show that the locus of the 
intersection of the normals at local symmetries is an ellipse. In region II the locus is 
a hyperbola. Figure 22b shows the smoothed local symmetries of the pliers (Figure 
21 ) computed by comparing arcs directly. 


4. Representing significant changes in curvature 

There are two reasons for discovering and representing the significant changes 
in curvature along a contour. First, they provide a set of knot points for constructing 
a perceptually close approximation to the contour. Second, as we pointed out in 
Section 2.1, smoothed local symmetries are both a region based and contour based 
representation of shape. A contour based representation requires that the curvature 
changes be detected, localized, and described correctly. This section sketches a 
representation of the significant curvature changes along a contour, a representation 
that we call the curvature primal sketch. Further detail can be found in [Asada and 

Brady 1984]. 

Our approach follows that of Marr [1974]. We take ideal parameterized versions 
of a particular set of curvature changes as our models /*($), where /,• is described 
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Figure 22. a. Geometry for computing the smoothed local symmetry from two circular 
arcs. b. The smoothed local symmetries of the pliers computed by directly comparing pairs of 

circular arcs._ ________ 

as a function of the orientation of the tangent to the contour. We then determine 

analytic forms for G" a * /»(«), where 

1 t 2 

Ga{t)=— r-exp-r-j 
y/2Ha la 

is a Gaussian of standard deviation a. We use Gaussian second derivatives Gj 
(approximately equal to the difference of Gaussians [Marr and Hildreth 1980]) 
as filters because of the analogy between curvature and intensity changes. The 
algorithm filters the contour with a set of Gaussians of increasing standard deviation 
(scale) to construct a scale space representation of the contour [Witkin 1983, Yuille 
and Poggio 1983]. Using the analytic form of G" * fi as a guide, we construct a 
program to find instances of the curvature change model along actual contours. We 
have identified a set of models, developed the analytic form of their convolution 
with the second differential of a Gaussian at several scales, and implemented an 
algorithm that matches models against the differentiated curvature along an actual 

contour. 

Figures 23, 24, and 25 show the set of ideal curvature changes that form 
our current set of models. There is an exact analogy between the problem of 
representing the significant curvature changes along a contour and the “primal 
sketch” representation advocated by [Marr 1982] for significant intensity changes 


23 




Hrady&Asada 


Smoothed I<ocal Symmetries 





(C) 


Figure 23. a. The corner model consisting of two circular fragments of curvatures and 
* 2 enclosing an angle <f>. b. The corner model in orientation space, relating the orientation of the 
tangent to the curve to distance along the curve, c. The result of convolving the corner model 
with the second derivative of a Gaussian. _ _ 




3 


(O) 



Figure 25. The remaining models in orientation space, a. The end b. The dent or bump. c. 
The smooth join , which is often difficult to detect o r localiie. ___ 

in a single direction across an image. For example, the corner shown in Figure 23 
is analogous to a step change in intensity, while the crank shown in Figure 24 is 
analogous to a thin bar. 

Consider the corner fragment*,’defined (Figure 23b) by 


r corner 


M =.{ 


K\8 + C 
K2 * -1- C + ^ 


if 8 < 0; 
if s > 0. 
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Figure 24. a. The crank model consisting of three circular fragments of curvatures K lf k 2 
and K 3 enclosing angles <f>\ and <f> 2 , the radius of the crank being 2a. b. The crank model in 
orientation space c. The result of convolving the crank model with the second derivative of a 
Gaussian where oja > 1. 

It is tedious, though straightforward, to show that 

-6 s 2 8 2 

\/2 no(Ga * fcorner) 11 (&) = ^ 8 ^ _ K ') eX P( _ ^2 )' 

As Figure 23c shows, this function has a zero crossing near the corner separating 
two peaks of opposite sign. The distance along the s-axis between the zero crossings 
is 


<>corncr 


= -^V(* l-*2) 2 + 


4 <t> 2 


The formula for the height of the side lobes is complex. In the special case *i = * 2 , 
it reduces to 


, 1 w 

•Icorntr — o 


Figure 26 shows some typical variations 


of the corner model. 
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Figure 26. Variations of the corner model. ______ 

As a second example, consider the crank model, whose analytic form (see 
Figure 24b) is 


fcrank { s ) 


*i(s + a) + C 

/C 2 (s + a) + c 4* <f>\ 

k,z(s — a) + (2a/C2 + c + <f>\ + fa) 


if 8 < —a; 
if — a < 8 < o; 
if 8 > a. 


The crank is analogous to a “thin bar” intensity change in the intensity primal 
sketch. It is even more tedious to show that 



+^ 2 (« - a) exp - 

+ (*3 - *2)exp 
+ (*2 - *l) exp 



The qualitative form of the function [G a * /eranJt)"M depends on the relative values 
of its parameters, in particular the degree of smoothing a and the contour length a 
of the step of the crank. Figure 27 shows typical variation when K\ = * 2 = *3 = 0 . 
When the ratio u/a is less than' 1/2, the crank is essentially signalled by two 
independent corner responses. However, when a/a is greater than one, it produces 
a strong central peak with two side peaks of opposite sign that are at most half 
the height of the central peak. .* 
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Figure 27. Second derivative of a crank after Gaussian smoothing. The edges of the crank are 
assumed to be straight. The responses vary according to the ratio a/a, the exact values being 


shown to the side of the figures. 



Figure 28. Response of a straight line end as it varies with a/a. We assume that each ic,- 
in Figure 23 is xero, and that the <f>j = w/2. The responses are for ofa = 0.5,0.75,1.0,1.25. 


We have similar analyses for the three models shown in Figure 25. A smooth 
join generates a peak of height (^ — rtiVC^V^Tr). Only when the curvatures flanking 
the smooth join are very different can this be reliably found. This is consistent with 
human perception of such smooth joins. Figure 28 shows the second differential of 
a Gaussian convolved with an end whose edges are straight. As in Figure 27, the 
response varies with a /a. Finally, Figure 29 shows the response to a bump. If the 
bump is long enough, the response is equivalent to two separate edges of opposite 

sign. 
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Figure 20. Response of a bump or dent. The bump in the case shown here is circular, with 
initial and final angles ir/ 4. _ • _ 

Figure 30a shows the second derivative of the smoothed orientation at a variety 
of scales a along the contour of a hammer. The model instances described above are 
marked on the graphs shown in Figure 30b. In Figure 30c we show the knot points 
that mark the significant curvature changes along the contour of the hammer at 
the different scales. Finally, in Figure 30d we show the smoothed local symmetries 
resulting from applying the approximation algorithm described, in the previous 
section given the knot points shown in Figure 30c. 

5. Examples 


In this section we show some examples of the smoothed local symmetries 
computed for a set of hand tools. Figure 31 shows a screwdriver. The global axis 
of symmetry is found by the algorithm. Note that at the coarsest scale, the two 
corners forming the end of the handle, and the (primitive) end of the blade, axe 
discovered, as are the two cranks defining the join of the handle and blade. Figure 
32 shows a tack hammer. The head of the hammer, consisting of the two striking 
surfaces, is easily found. So is the handle. The join of the handle and the head is 
a side-end join, and can be found using the method sketched in the next section. 
Figure 33 shows a carving knife (compare Figure 17) and a cheese knife. 


6. Finding sub-shapes 


The shapes that populate our world have identifiable sub-shapes. A hammer, 
for example, has a head and a handle. The head consists of a striking part, that has 
a flat surface, and a counterweight part, whose shape varies considerably between 
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Figure SO. a. The/second derivative of the orientation along the contour of a hammer, 
smoothed at a variety of scales, corresponding to a — 5,8,11,22. b. The model instances are 
shown in a tree diagram as they arc tracked over the different scales, c. The knot points that 
correspond to the model instances in (b). d. The smoothed local symmetries resulting from 
applying the approximation algorithm described in the previous section given the knot points 
shown in (c). 

hammers. Our current work aims to make sub-shapes explicit. In this section, we 
report our progress. Figures 34 through 36 show the representation that we axe 
aiming at. In Figure 34 we show how sub-shape information at a single scale, 
indicated by the part-of links, is integrated with the smoothed local symmetry 
representation discussed in the previous sections. Figure 35 shows how a shape can 
be represented at several scales. 

Figure 36 suppresses geometric detail to show other kinds of relationships, 
particularly part-of relationships, a-kind-of specialization relationships, and con¬ 
straints between elements of the. representation. A-kind-of links indicate class 
constraints. Instance links enable-us to index model parts from smoothed local 
symmetry representations of shape fragments. Other attributes of a shape are under 
investigation. One such is a set of motor programs that suggests for a model such 
as a hammer how it might be used and for what purpose. Part-of and a-kind-of 



















































































relationships have been explored extensively in knowledge representation studies 
in Artificial Intelligence. In particular, ACRONYM [Brooks and Binford 1980] is 
a major contribution to the development of representations of object models, and 
it has heavily influenced our work at the level indicated in Figure 36. All parts 
in ACRONYM models are (simple instances of) generalized cones, whereas in our 
work they are smoothed local symmetries, as shown in Figures 34 and 35. Following 
ACRONYM, we incorporate constraints, in the form of bounds, between links. For 
example, there are bounds on the relative sizes of the head and handle of a hammer. 
It would be surprising to find a sledge hammer with a two-inch long handle, or to 
find a half-inch wrench with a six. foot handle. An important part of ACRONYM 
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Figure 33. The smoothed local symmetries found for a carving knife and a cheese knife. 



Figure 34. The representation of the shape of a hammer. The integration of part-of links 
defining sub-shapes with the smoothed local symmetry representation of the contours of and 
regions subtended by the sub-shapes. 

————— ii ' —ii » '' ■' * ~^^ 

was the program of Brooks [1982] for determining bounds on expressions given 
bounds on their variables. 

We have not yet investigated .the incorporation of yet other sorts of information 
into our models. Such information might make explicit surface markings or textures, 
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hammer 




Figure 35. The hammer is represented at several scales, here two. At the coarsest scale, 
the handle is simply represented as a symmetric shape with constant cross section. At the finer 
scale, the shapes of the handle and head are refined. 

for example. As we noted in the Introduction, in practical applications, the model 
database could be very large. 

Our representation proposes four major modifications to the ACRONYM 
representation. First, we use smoothed local symmetries instead of generalized 
cones. The reasons for this are developed in [Brady 1983]. Second, we represent 
a shape at multiple scales, for the reasons discussed in Section 5 and in [Asada 
and Brady 1984]. Third, we incorporate additional links, particularly instance 
links and motor programs , into the representation. Finally, unlike ACRONYM, we 
explicitly represent sub-shape connections (Figure 34). As we discuss more fully 
below, sub-shapes can be joined together in a variety of ways, in particular in ways 
we call side-end and end-end [Marr 1977]. 

In general, deciding that a portion of a shape is a sub-shape requires the 
mobilization of application-specific, semantic information. Here, we are concerned 
with lower level processes, embodying weaker semantics, that nevertheless isolate 
sub-shapes in a usefully wide class of instances. Several methods have been proposed 
to determine the sub-shapes of a shape. 

For example, the original motivation for the concept of branch point in the 
symmetric axis transform was to determine sub-shapes. The idea does not work 
well in practice. First, as we noted in Section 2.2, branch points often occur when 
there is an orientation discontinuity on the contour, in which case they do not signal 


32 




Itrady&Asada 


Snioolhcd l/oc&I Symmetries 



Figure 36. Other relationships of the representation. A-kind-of links indicate specializations 
such as class inclusion. Function links are under investigation. 


sub-shapes. Second, branch points consist of a single point that is the center of a 
particular circle, and so their placement is highly sensitive to noise on real contours. 
In other work, Nevatia and Binford [1977] located large changes in the orientation 
of the axes of generalized cylinders to determine sub-shapes. The algorithm would 
fail to find the join between the handle and blade of a screwdriver, for example, 
because the (symmetry) axis is straight. Marr [1977] proposed that object joins may 
be classified as side-end or end-end, though the notions of side and end were never 
formally defined. Assuming that the objects to be joined were generalized cones 
with straight axes, he developed a geometric analysis of a side-end join [Marr 1977, 
Theorem 6 and Figure 23], and proposed a formal version of the familiar matched 
concavities heuristic to find side-end joins. The test he proposed required that the 
angles at the joins sum to 7r. The approach was not implemented. It would be as 
sensitive to noise as the branch point approach of the symmetric axis transform. 
Many authors have proposed that side-end joins can be detected by finding a pair of 
concavities in the shape, each indicated by a negative minimum of curvature. This 
idea does not work in practice. First, unless the contour is smoothed by a suitable 
filter such as the difference-of-Gaussians, negative minima of curvature may be 
caused by noise, giving rise to false positives. Second, the absence of curvature 
maxima may give rise to false negatives. Figure 37 shows that finding a pair of 
concavities is a sufficient, but not necessary, condition for asserting a subpart of a 
shape. 

° • • 

In order to find the sub-shapes of a shape, let us ask first how two shapes can 
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Figure 37. A aide-end join, one of whose concavities b not signalled by a significant negative 
minimum of curvature. 




Figure 38. a. Many shapes have alternative descriptions. The shapes shown on the left and 
right are typically interpreted as shown; the center shape b ambiguous, b. A related problem 
concerns ambiguity about what is joined to what. 


join. We postpone investigation of several important questions, such as: how to 
choose among different possible “parsings” of a shape into sub-shapes (Figure 38a); 
and how to determine which is the sub-shape and which the main shape (Figure 
38b). Figure 39 shows our model of sub-shape formation: two shapes may overlap 
with optional smoothing of their contours. Alternatively, the shapes may be close 
and their contours smoothly joined. Notice that it is the smoothing of a contour 
near a real subpart join that invalidates naive expressions of the matched concavity 
heuristic. 
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Figure 39. Model of (side-end) join formation. The contours of two overlapping or nearby 
shapes may be smoothly joined. 

Terminations 

In order to assert subparts, we need to define terminations of a shape, so 
that we can distinguish side and end. In our approach, a shape is represented 
by a (finite) smoothed local symmetry. By definition, a smoothed local symmetry 
axis is a maximal locus of local symmetries. It follows that a smoothed local 
symmetry axis partitions a shape into two sides , which are those portions of the 
contour that generate the axis, and two terminations , which are the remaining parts 
of the contour. Switching attention between different smoothed local symmetries 
correspondingly changes the parts of a contour that are perceived as side and 
perceived as end. Sides are typically elongated, which is an alternative way of 
saying that, all other things being equal, we prefer descriptions based on a shape’s 
longest axes. Notice that the description of a join is not a symmetric relationship 
between sub-shapes. In Figure 40, the "handle” A is on the side of B but B is on 
the end of A. Such alternative descriptions can be extremely useful in adequately 
characterizing a join. 

Currently, we distinguish four types of termination (Figure 41). A termination 
is considered open if it is the distal end of a negative smoothed local symmetry 
corresponding to a concavity. Typically the contour “turns away” at an open 
termination. All other terminations are closed , and are of three types. First, in 
a tapered termination (Figure 41a) the width function h(s) slowly tends to zero. 
Typically, the symmetry angle a is nearly constant, and a tapered termination gives 
rise to a (single) curvature discontinuity of type end. Second, for a blunt termination, 
the width function tends to zero but “quite abruptly” (compare Hollerbach’s [1975] 
and Heide’s [1984] use of symbolic qualifiers corresponding to ranges of values). 
Typically, there is an abrupt chartge in the symmetry angle a. A blunt termination 
usually generates a curvature discontinuity that is an end at small scales but a 
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Figure 41. Three types of closed termination of a smoothed local symmetry, a. tapered, b. 
blunt, c. abrupt. 

pair of corners at larger scales. Note that both tapered and blunt terminations are 
signalled by curvature discontinuities of type end. Finally, an abrupt termination 
(Figure 41b) gives rise to a pair of curvature discontinuities of type corner. In 
general, nothing can be asserted about h(s) or a. Of course, there is no absolute 
distinction between a blunt and tapered end, or between a blunt and tapered 
end. Positive smoothed local symmetries (“figure”) have two closed terminations. 
Negative ones (“ground”), corresponding to concavities in a shape, have an open 
distal termination and a closed proximal one. 

* 

Subpart Joins 
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We propose two rules that assert the existence of a subpart join, prior to 
determining whether the join is of type side-end or end-end. As we have noted, 
and as we shall see further below, this is because the notions of side and end are 
intimately associated with (the global property of) elongation. Deciding that part 
of a shape is side or end derives from choosing an sis to organise one’s description 
of a shape. 

.The first rule generalizes the familiar matched concavity heuristic: 

Rule 1 

if there are two corresponding concavities, 

then the cover of the sis associated with the concavities is a subpart. 

The crucial problem is how to assert the existence of concavities. There are various 
possibilities, and they rely on the fact that our representation is both contour and 
region based. First, there may be a significant curvature change of type corner. 
Second, there may be a primitive curvature change of the crank type that indicates 
a (small) concavity. This suffices, for example, to isolate the handle and shaft of a 
screwdriver. 

Finally, a concavity can be asserted even if there is no significant curvature 
change, if the contour is corrupted by noise, or if there is a significant curvature 
change that is overlapped. In such cases we rely on the region component of 
our representation. A concavity is a “negative” shape that has a smoothed local 
symmetry axis. Figure 42a shows a matched pair of concavities and the graph of 
contour fragments and smoothed local symmetry axes to which it gives rise. We 
assert the existence of a subpart, whose sis is \l if (i) we find an instance of the 
sub-graph shown in Figure 42a; and (ii) the contour fragments a and d are smoothly 
extensible without a significant change in curvature. 

There is an analogous rule for matched convexities asserting the existence of a 
negative subpart such as an inlet (see Figure 42b). For example, the axes X and u in 
Figure 42a, corresponding to the concavities, have been labelled with a minus sign. 
Notice that reversing the sense of figure and ground corresponds to interchanging 
minus and plus. Figure 42b is interpreted as a subpart that is a negative shape, 
that is, an inlet. 

Suppose that Rule 1 is applicable. The subpart it defines is called a limb if it 
is elongated and has a closed termination. If it is not elongated, it is described as 
an attached subpart. Finally, if it does not have a closed termination it is called a 
subpart join. Figure 43 shows examples. 

Rule 1 is sufficient but not necessary for asserting the existence of subparts. 
Figure 44 shows three example shapes in each of which there is only one concavity. 
Figures 44a and 44b are end-end joins, but Figure 44c is the side-end join shown 
in Figure 37. Figure 45 shows once more the crucial role that elongation plays in 
asserting the existence of a subpart. The fragment shown in Figure 45a is perceived 
as a join in Figure 45b but not in Figure 45c. 

Rule 2 
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Figure 42. a. A concavity is asserted if there is an instance of the sub-graph of the 
smoothed local symmetry representation shown on the right, b. Interchanging plus and minus 
enables us to assert negative subparts such as “inlets”. 



Figure 43. a. A limb. b. an attached subpart. c. a subpart join. See text for details, 
if there is a concavity, 

and there is an sis, one of whose generating contours flanks the concavity, 
such that the sis is either elongated or has a closed termination, 

then the cover of the sis is a subpart. 

If the subpart has a closed termination, it is described as a limb if it is elongated 
or an attached subpart if it is not. Otherwise the (elongated) subpart is called a 
join. Figure 44 shows examples of, object joins. As in the case of Rule 1, there is 
an analogous rule for convexities that signal inlet subparts. Other rules are under 
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Figure 44. Matched concavities provide a sufficient, but not necessary, for asserting the 
existence of subparts, a. an end-end join. b. an end-end join generated by a crank, c. the side-end 
join shown in Figure 37. 



Figure 45. a. a shape fragment, b. the shape fragment in (a) corresponds to a subpart join, 
c. the shape fragment in (a) does not correspond to a subpart join. 

investigation. 

Recursive application of the rules 

When a subpart is asserted, the smoothed local symmetries of the two parts 
are then (re-)computed. (Of course, there are efficient ways to use the existing 
information.) The crucial point is that we inhibit local symmetries between points 
that are from different sub-shapes. This removes many apparently spurious axes from 
the initial investigation of a compound shape. The process is applied recursively. 
This generates a further hierarchical shape organisation that is similar to that 
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Figure 46. a. A notched rectangle, b. The smoothed local symmetry axes for the notched 
rectangle, c. The result of inhibiting spines from the rectangle and notch shapes found by rule L 

advocated by Marr and Nishihara [1978]. 

Consider, for example, Figure 46. It shows the notched rectangle, discussed by 
m any authors in connection with the symmetric axis transform. Figure 46b shows 
the smoothed local symmetry axes computed for the notched rectangle. There are 
several small axes that seem to be spurious. There is also a negative inlet (triangle) 
shape that can be found using Rule 1. Crucially, every spurious axis derives from a 
portion of the contour of the inlet subpart and a contour of the rectangle. Inhibiting 
them leaves precisely the description: an inlet into a rectangle. 


7. Conclusion 

We have introduced smoothed local symmetries as a representation of two- 
dimensional shape that is both contour and region based and which has local 
support. We described two implementations, one of which efficiently computes an 
approximation to the smoothed local symmetry representation, and relies upon 
computing the curvature primal sketch representation of the significant curvature 
changes along a contour. The previous section reported some initial findings on the 
isolation of subobjects. 
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Currently we are developing a database of parametric models of objects that 
enable them to be recognised even when they are occluded. We are investigating 
the relationship between function and form to support reasoning about objects. 
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